1. Field of the Invention
The present invention relates to a photo retrieval method, system, and medium using multimodal information which prevents a photo of an irrelevant category to be retrieved as a high ranking due to an unfavorable side effect from a query expansion, by reflecting a level of consistency between an object category of a query and a category of a photo retrieved by expansion retrieval using ontology, to a photo retrieval result ranking.
2. Description of the Related Art
Popularization of digital cameras and camera phones cause many changes associated with using a photo. Unlike making a print of a photo on paper and semi-permanently storing the photo, in a digital camera, more images are easily photographed, deleted, and stored electronically. Currently, as memory capacity increases, an amount of storing photos increases in proportion. A user may store more photos due to an increase in the amount of storage available for the photos. However, storing more photos increases the difficulty of arranging photos or retrieving a particular photo.
A conventional photo retrieval method depends on retrieval by simply browsing or manual arrangement. It is not easy to retrieve several hundreds or thousands of photos by simply browsing.
To improve a browsing retrieval method, a configuration performing photo retrieval by expansion retrieval using ontology is disclosed. Query expansion by ontology has an advantage of improving a recall rate, but has a disadvantage of decreasing precision. This is because a query term that is not suitable due to ambiguity is extracted as an expanded query term when expanding a query.
For example, in a conventional ontology expansion retrieval method, with respect to a query “friend”, “friend, mate, fellow, and companion . . . ” are extracted as expanded query terms. However, an error of retrieving a photo associated with “mate to the glove” that is irrelevant to the query “mate” may occur by using the expansion retrieval method.
Also, an error in retrieving a photo associated with “garden rank with weeds” as a retrieval result may occur by analyzing a query “rank (line)” as “rank (dense)”.
Also, applying a method frequently used in conventional text-based retrieval methods to photo retrieval may prevent expansion to unnecessary query terms by analyzing a meaning of an ambiguous vocabulary by using peripheral contexts. The method using text analysis is suitable when retrieving a text file such as in a general retrieval from the Internet. However, when retrieving a photo including a short annotation, context information for analyzing a meaning may be insufficient. Also, since a keyword included in the annotation is not always suitable for the photo, there is a limit on retrieval by using only the text information.
Accordingly, a photo retrieval method, system, and medium using multimodal information, which can prevent an irrelevant photo from being retrieved, is acutely required.